1. Field of the Invention
The invention refers to an interactive computer system and methods for the acquisition and processing of the bio-kinetic information of the signature for the assessment of dynamic handwritten signature authentication. It has applicability in the field of behavioral biometrics, in situations where there is user interest to validate his will by a handwritten signature.
2. Description of the Related Art
It is generally accepted that, in the procedures of declared identity validation of a person, in addition to the administrative, physiological biometrics, computer based methods and technologies, there can be used, as a supplementary link, other methods and technologies that belong to behavioral biometrics field. The acquisition and recognition of handwritten signature constitutive elements represent a class in behavioral biometrics.
In the RO 141297 and EP 1846868 patents, described here by reference, there are already developed an acquisition solution and processing methods of the dynamic information (accelerations) associated to the signing process in order to verify the handwritten signature. The information is captured, processed and compared by a computer-based system. The acquisition of acceleration signals by inertial accelerometers—MEMS, integrated into an electronic pen, and the specific topology of placing the sensors, facilitate not only the acquisition of spatial-kinetic information, but also the acquisition of contact microvibrations generated at the paper level. The processing methods on acquired acceleration signals determine:
the starting and finishing points of the signature, a method realized through algorithmic data processing, which represents the variation in time of a threshold distance to the paper combined with the variation of the frequency and amplitude parameters of the contact microvibrations generated by the subject/pen/paper interaction,
Invariant sequences associated with the initial acceleration signals and invariant sequences associated with the derived components of initial acceleration signals.
the distance between two signatures by algorithmic comparison of the sets of invariant sequences using two different methods.
the final result is determined through a decision method in which targeted subjects' samples and other subjects' samples registered in the signatures database take part through the results of comparisons with the input signature.
The system and methods in RO 141297 and EP 1846868 patent deal with special kinetic phenomena combined with contact microvibrations by processing the accelerations captured by the accelerometers—MEMS. However, this process has the disadvantage of not including both the capture and the processing of graphic information which, if combined with kinetic information by sensorial fusion, it would increase the system's accuracy and the quality of man-machine interaction. The present invention aims to eliminate the disadvantages mentioned by incorporating, along with the inertial accelerometers—MEMS, an optical navigation sensor, considering the conditions of the sensorial fusion concept. This optimization is used graphically and kinetic, on the one hand, in the new algorithmic, multimodal processing methods of the captured signals, and, on the other hand, for the achievement of a visual feedback for the user along with the advantages described, further on, in detail.
U.S. Pat. No. 7,176,906 B2 (Microsoft Corporation) patent presents a method of interpreting the handwriting thickness using electronic methods based on ballistic movements. In order to capture the information, it is used a pen with (incorporated) accelerometer/accelerometers. For proper interpretation of the information, it is used “the pulse width variation provided by the accelerometer or the pen's angle of inclination.” This method implies the use of a pen that contains accelerometers. However, it does not contain any element for the direct description of its graphic trajectory. The method suggests, in order to associate the information regarding the handwriting thickness with the trajectory of the pen, the use of complex and expensive external devices of graphic tablet type. The specific issues of handwriting signature are not discussed there.
In contrast with the above-mentioned patent, the present invention contains a signature capture module of pen type, which integrates, by sensorial fusion, not only accelerometers, but also an optical navigation sensor having the purpose of capturing the graphic trajectory. The topological concept of sensor integration in the pen is essentially defined to guarantee the spatial synchronism of the accelerometers' axes with the axes of optical navigation sensor. The functional integration of both categories of sensors in the pen ensures the synchronization in time of the acquisition of the two types of data: accelerations and self-referential movements. The information acquisition's space-time synchronism is the principle that governs the sensorial fusion, whose application in this invention produces a better accuracy in the signature authentication. The function of thickness handwriting interpretation is defined and based, not only on the variation of contact microvibrations frequency, but also on their amplitude variation as an effect of dynamic pressure that occurs between the pen and paper in handwriting.
U.S. Pat. No. 7,433,499 B2 (DynaSig Corporation) patent describes a signature authentication and acquisition system which contains accelerometers and pressure sensors incorporated in the writing device, the authentication method being achieved by data encryption and by comparing code results, thus avoiding the storage of unencrypted samples. The patent does not deal with any information about the graphic nature of the writing device or the authentication method.
U.S. Pat. No. 7,483,018 (Microsoft Corporation) patent describes both a system and a dual method for graphic data input, of pen type, in which the reference pattern, analyzed by the integrated image sensor, is the source that generates absolute or relative coordinates, depending on the implementation option and the context of use. System generates graphic or position data, but it does not include components and methods for handwritten signature authentication. The fact that the accelerometers are not integrated in the capture module (in this case, a pen) reduces the applicability in signature authentication.
U.S. Pat. No. 7,508,384 B2 (Data Research Inc.) patent describes a system and a method of acquisition and recognition of hand movement when writing on a surface—“Digital Writing System.” Kinetic spatial data capture is performed using a 3-axis gyroscope, a 3-axis accelerometer integrated in the pen and a proximity sensor related to the writing support. The patent does not describe capture sensors for graphic information nor algorithmic methods of signature authentication. 3D trajectory data are estimated indirectly by dynamic linear filtering based on Kalman estimator (filter).
The procedures requiring handwritten signature verification are outlined and applied using concepts, principles and customs belonging to social and legal sciences, behavioral psychology and human neurophysiology. Electronics and information technology contribute to improve the security of checking procedures by means of acquisition, processing and recognition of signals and signature forms.
The usability, the accommodation and acceptance degree from the user of the biometric procedure are the essential elements, defined in literature [http://www.biometrics.org], used as arguments to the classification of biometric technologies. Most people would prefer the biometry of the handwritten signature to other biometric methods. The signature is, first of all, considered a personalized and individualized means of self-protection due to its social utility and personal perception. The justification lies in the individualized nature of the motric phenomenon produced in handwriting—an acquired reflex, linked to self-interest and free will. The fact that the present invention deals with the acquisition and processing of signals specific in the psychomotricity of the signature, by maintaining the paper as a support, or any other material with similar texture, offers a high degree of acceptance and accommodation.
While making a signature, the user's forearm stands on elbow, on a horizontal desk so that only the palm and the fingers perform the spatial and dynamic gesture associated with the production of the signature. In this context, a self-referential optical navigation sensor (ONS), located in the pen, quasi-parallel with the writing plan and near the tip of the pen, has, as reference for navigation, quasi-uniform spread elements of the pattern printed on paper/support. The lens (L) captures the dynamic image of pattern's elements by projection on the photosensitive area of the optical navigation sensor. The visual field of the lens covers a quasi constant area that contains sufficient pattern elements so that the ONS sensor to determine the relative movements (reported to its own coordinate axes x, y) from a previous time moment. Based on the differences between consecutive images, internally sampled by ONS from the continuous image, the sensor generates pairs of relative movements dx, dy, at a constant period of time, in a milliseconds scale, controlled and synchronized by the microcontroller placed in the pen, with the capture and the analog to digital conversion of acceleration signals (see RO 141297 and EP 1846868), sensed by MEMS A and MEMS B sensors. As a result, a computerized graphic representation of the ONS sensor trajectory is similar to the graphic made by the pen's refill tip. The graphic differences between the representations of ONS trajectory and the graphics made by the pen's refill represent a morphism of the same motric phenomenon, caused by the dynamic inclinations of the pen. The morphism individualizes biometrically the signature. Moreover, due to the synchronization between the graphics done on paper and the trajectory's capture through the ONS sensor, the sequential events that make up the two representations—the trace left on paper by the pen's refill and the electronic trajectory captured by the system—can be electronically stored, including the trajectory's segments where the pen's refill tip does not touch the writing support. In these situations when the execution of the signature implies momentary liftings of the pen from the paper and the amplitude of these liftings is small enough, of millimeters or even smaller, the optical navigation sensor continues to focus on the pattern elements printed on the writing paper and also continues to capture the trajectory. System continues to acquire the information produced in these sequences (when the pen does not touch the paper) because it contributes to the signature's individualization. This information is part of the signature and is handled by the processing and graphic display module, firstly, by displaying it in order to achieve visual and kinetic apperception by the user and, secondly, by calligraphic screening and processing as segments that do not need display in order to achieve graphical view.
Handwritten signature is a psychomotric act. Psychomotricity is defined as being the result of the integration of motric and mental functions, under the effect of the nervous system's maturation, and regards the report between the subject and his body. Psychomotricity is therefore not only an ability, but also a complex function adjustment of the individual behavior, including the participation of various psychomotric functions and processes that provide both data reception and proper execution of the response act.
As J. Piaget [Theory of Cognitive Development—1952] claims, the motric and the psychic are not two distinct categories, one subject of pure thought and the other subject of physical and physiological mechanisms, but the bipolar expression of a single process, namely that of efficient and flexible accommodation to external conditions. Between the external conditions of the motric act and the subjective conditions, the act of signing is not only a mechanism of execution, but also a loop circuit in which every stage, every detail of the operation is the immediate expression of the relations established between individual and environment, respectively between individual's will, psychomotrically manifested by semi-acquired reflexes, and the support of will's projection, in this case, the writing paper, where the signature is performed.
The fact that a signature is produced with a writing device, which is practically one with the subject during performance, guided the inventive principle towards the capture of the motric variables principle (accelerations and relative movements) by self-referential sensors incorporated in the pen (the writing device), thereby emulating the non-intruding integration of the sensors “in the subject”. The self-referential concept transposes into practice the need of a close connection between the observer (sensors) and the psychometric act, thus capturing the data from subject's sensory motor perspective, in the context in which the purpose is to observe and to assess the individuality of the psychomotric act—the subject's signature. The writing paper has standardized qualitative properties, relatively constant, which is why the expression of individuality through the signature's motric act will be affected in a low extend by the potential differences in the quality of the paper.
The set of trajectory's attributes, captured by ONS sensor, along with the sequencing and spatial gestures captured by the accelerometers MEMS, are elements that, made by subject's free will, individualizes the signature.
From the user's point of view, the role of integrating the visual feedback into the system is dual:
it favors the cognitive mechanisms of gestical apperception, providing the user with the possibility of acquiring, through practice and visualization, complex gestures, unwritten on paper, but belonging to signature's kinetics. The user can deliberately create these acquired gestures for an additional individualization of the signature,
it facilitates the user's accommodation with the specificity of the system, due to the combination between psychomotric mechanisms and display.
The consistency and complexity of the signature's gestures, naturally acquired in time or through practice, are limited only by the motric abilities and by user's ingenuity to compose, by gestical elements, a signature as individualized as possible.